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ABSTRACT 

Components that should comprise a model for analyzing 
the impact of educational reform are discussed. An analytic framework 
is presented that consists of a synthesis from research of the 
following elements considered crucial in the formation of such a 
framework: (1) objectives or goals of the reform; (2) processes used 
to reach those objectives; ana (3) outcomes that result from 
interaction of the objectives and processes. These categories are 
placed and analyzed in a broader social, political, economic, and 
cultural context that in turn is placed within an ideological context 
or theoretical perspective. Positivism, interpretivism, and critical 
theory are compared; and critical theory is identified as the basis 
of the proposed analytic framework. The next layer of the model 
explores contextual considerations. A model must also include an 
analysis comparing the objectives of the reform to the processes by 
which the reform is conceived and implemented. A model based on 
critical theory supports the necessity of using standards of societal 
concerns to assess the degree to which educational outcomes have been 
achieved. Assessing this degree of achievement is accomplished 
through an analysis of specific policy goals and outcomes and then 
through questioning standards and criteria that had previously been 
accepted as givens, such as equity and quality- of life. (Contains 62 
endnotes and 32 references.) (SLD) 
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Introduction 

This paper is a theoretical discussion of those components 
which should comprise a model to analyze the impact of education- 
al reform. While a multitude of evaluation models exist, they are 
often limited in that they focus on one aspect of evaluating 
reform at the expense of others. For instance, models which 
emphasize the initiation of the reform may not look at dissemina- 
tion or implementation of the reform. Models looking at reform 
from the perspective of what goes on within classrooms may not 
include an analysis of the contextual variables within which the 
reform is situated. Or models utilizing a simplistic linear 
analysis may merely compare inputs of the reform to educational 
outcomes . 

Consequently, models or analytic frameworks for assessing 

educational reform generally do not include a wholistic analysis 

of all the necessary variables for evaluating impact. It becomes 

difficult, then, for federal, state, and local policymakers to 

make wise choices regarding appropriate models for effective 

implementation or assessment of reform efforts. In critiquing 

models for evaluating education reform Larry Cuban says 

it is still not clear which implementation 
strategies are most effective because few 
researchers have investigated the connections 
between strategies and outcomes* Most re- 
searchers have recognized the tangled com- 
plexity of such diverse elements as context, 
roles , individual , organizational factors 
such as size, history, and culture, quality 
of leadership at both district and school 
levels, timing, and other critical determi- 
nants of successful implementation. Beyond 
recognizing this complexity, few researchers 
nave proceeded further than to construct 
inventories and taxonomies of essential 
points in establishing causal relationships 
between strategies and outcomes.^ 

Thus, the following analytic framework for evaluating the 
impact of education reform consists of a synthesis from the 
research of those elements considered crucial in the formation of 
such a framework. Three separate categories: (1) objectives or 
goals of the reform, (2) processes used to reach those objec- 
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tives, and (3) outcomes which result from an interaction of the 
objectives and processes form the basis of this freunework. Thf»se 
three categories are placed and analyzed within a broader social, 
political, economic, and cultural context which in turn is placed 
within an ideologic context or theoretical perspective. The 
resulting frcunework is schematically represented on page 23, 
Analytical Framework for Analyzing Impact of Education Reform 

Ideological considerations . The first consideration, as 
shown in the outer layer of this model, needs to be an acknowl- 
edgement of the social science paradigm which influences and 
guides one's work as this will have an influence on the ideologi- 
cal perspective and ultimate values forming the basis for the 
model. According to Wilfred Carr and Stephen Kemmis, 

paradigm embodies the particular conceptual framework 
through which the community of researchers operates and 
in terms of which a particular interpretation of 'real- 
ity' is generated. It also incorporates models of 
research, standards, rules of enquiry and a set of 
techniques and methods ,.. .consistent with the view of 
reality that the paradigm supports.^ 

Most evaluation models, especially those following empiri- 
cal-analytical rules or a sub jectivist-relativist position, have 
traditionally overlooked the importance of acknowledging a 
theoretical base and value-orientation shaped by an underlying 
ideological perspective as Holland Paulston contends, "many 
unspecified theoretical and ideological axes are ground in 
educational change and reform studies, but, unfortunately, little 
of this is ever acknowledged or made explicit."^ Any model 
overtly ignoring ideology is lacking a crucial element as ideolo- 
gy helps form the basis of value systems and perceptims of the 
world, the nature of humankind and .society and relations of power 
within society. Messick (quoted by Paulston) asserts that 

what is at issue is ideology. It is not the implica- 
tions of research per se that are to be implemented in 
the proposed strategies, it is the implications of 
research as interpreted or filtered through a particu- 
lar ideology about the nature of man and society.* 

Thus, ideological perspective determines what values ought to be 
a part of any framework for evaluating education reform, influ- 



ences how the evaluation actually is conducted (the methodologies 
used)^ and impacts on the types of questions that are ultimately 
addressed regarding the focus of the evaluation. 

As evaluation practices are never value- free, one must then 
determine what values should guide the practice of doing evalua- 
tion to assess impact of the reform. According to Kenneth Sirot- 
nik and Jeannie Oakes "there are some values that we cannot 
negotiate away no matter what the evaluative problem or setting 
might be."^ They believe the values of social justice and social 
equality are paramount and must be at the heart of evaluative 
inquiry to use as a standard for interpreting whatever empirical 
data might emerge. Taking these values into consideration a 
brief discussion follows comparing and contrasting positivism, 
interpret ivism, and critical theory. 

(1) Positivism . The positivist paradigm has traditionally 
dominated the field of educational evaluation • Rationale for this 
model comes from the premise that science allows us to control 
the natural world, so it should therefore also allow us to 
control education and make it more congruent with the needs of 
society and its members.^ While positivism is assumed to be an 
unbiased, value-free, objective view of society and the world, an 
historical analysis by Thomas Kuhn reveals that a positivist 
conception of objective knowledge is nothing more than a myth. He 
argues that science needs to be seen as a series of revolutions 
in which the dominant paradigms are overthrown and replaced with 
new ones.^ This alone attests to the fact that a switch in para- 
digms means acceptance of new values and new beliefs. Consequent- 
ly, objectivity becomes hard-pressed to prove and the absolute 
logical distinction positivists make between theory and observa- 
tion is, thus, undermined. 

In an era calling for accountability and tighter control of 
the curriculum, evaluations conducted from a positivist theoreti- 
cal perspective have supported utilization of standardized tests 
as a means to judge the degree of fit between student behavior 
and learning objectives.® Accountability measures have been 



criticized by Thomas Popkewitz who believes these measures equate 
education to the specification of rrecise, measurable objectives 
or outcomes^ have a limited focus on learning mastery^ and ulti- 
mately result in improved efficiency and control of school 
organizations.^ Another limitation^ he sites ^ is the way in 
which positivism defines change processes in schools as nothing 
more than "a rational^ logical sequence^ subject to analysis and 
implementation through a science of management . "^^ He says^ 

accountability creates a map of the world that seems to 
provide an interpretation of everyday life. This... 
assumes consensus and diverts attention from the strug- 
gles^ strains^ and contradictions that underlie insti- 
tutional practices. . .making current objectives more 
concrete and measurable rigidifies existing content^ 
structure^ and value. 

( 2 ) Interpretivism . The goal of evaluation^ conducted within 

an interpret ivist theoretical framework^ is to understand the 

subjective meaning of reality as interpreted by the actors, 

themselves. Regarding interpretivism, Egon Guba and Yvonna 

Lincoln assert that 

what there is that can be known exists only in the 
rninds of people in the form of constructed realities. 
Each person has a different construction, shaped by 
such things as his or her experience, sophistication, 
opportunity to learn, and especially by his or her 
value system. 

The evaluator's responsibility is to report these subjective 
meanings to the intended users of evaluation. Doing evaluation 
involves gaining multiple perspectives of participants, insights, 
and meanings from the educational situation being studied or 
evaluated. Data collected is validated by cross-checking the 
different sources of data, comparing the various perceptions of 
the participants, and through negotiation, determining evaluative 
standards. "Evaluation knowledge, then, is defined as valid only 
to the extent that it reflects the consensus of the educational 
participants in each evaluation situation."^* 

As with positivism, the evaluator stands outside the evalua- 
tion situation "adopting a disinterested stance in which any 

o 



explicit concern with critically evaluating and changing the 
educational realities being analyzed is rejected, "^^ Reporting 
perceptions from the perspective of the participants, can thus 
result in politically conservative evaluations due to an uncriti- 
cal analysis of the historical and political context within which 
the evaluation is being conducted, "While it may be true that 
consciousness 'defines reality', it is equally true that reality 
may systematically distort consciousness.^^ This distorted con- 
sciousness or "false consciousness," as Habermas refers to this 
concept, means p€;ople fail to question their realities because of 
the effect of socialization which may prevent awareness of social 
situations that are contradictory, especially those with the 
potential to disrupt the existing harmony or status quo. 

(3) Critical theory . Critical theory, as a guiding theoreti- 
cal perspective for conducting educational evaluation, rejects 
positivist notions of rationality, objectivity and truth while 
legitimizing teachers' interpretive categories. In its critical 
orientation to situating practice politically and historically, 
it enables one to distinguish ideologically distorted interpreta- 
tions from those which are not. Although methodologically criti- 
cal theory outlines the study of particular kinds of problems, 
the aim is enlightenment and emancipation, it is not interested 
with what is but rather with what can and should be.^"^ 

While critical theory models of evaluation contain some of 
the same components as other models coming out of pcsitivistic 
and interpretivist paradigms, the differentiation focuses on the 
questions and values guiding the practice of the evaluation. One 
of the guiding tenets of critical theory is that it is embedded 
in questions regarding power relations in a society. Ignoring 
these relationships can ultimately result in the wrong conclu- 
sions being drawn. Landon Beyer and Michael Apple assert that 
every curricular evaluation process must ask prior questions 
regarding "evaluation for what social, economic, and ideological 
purpose?"^' Donna Coomer believes the questions upon which a 
critical evaluation is based, must reflect the relationships 



between educational change and concepts of power ^ knowledge^ 

ideology^ schooling and evaluation practices 

Critical theory has been criticized philosophically and from 

the pespective of practicality and relevance. While I acknowledge 

that criticisms of a philosophical nature regarding Habermas and 

his theory are in order^ discussion of those issues is beyond the 

scope of this piece Regarding practicality and relevance to 

real life experiences^ Young says^ 

perhaps the central theme... is that the historical 
self-relation of theory points to an ever present 
danger of constructing the interest in emancipation in 
too idealistic a form — in such a form it cannot be 
actually understood and practically carried out by real 
historical subjects.^^ 

Although ideals are an important and necessary part of theory^ 
there remains a need to apply them to practical circumstances. 
Thus^ a common criticlsn waged against critical theory especially 
by persons rejecting the legitimacy of its claims^ is its ideal- 
ist or Utopian stance. On the other hand^ intellectuals become 
disenchanted with the slow progress of practitioners in realizing 
change^ rather than recognizing that the crucial role of ideals 
is to help guide one's practice. As Young asserts, 

it is not the understanding of intellectuals which will 
carry forward actual changes, but the democratic pro- 
cess of many voices and practical problem-solving on a 
(lay to day basis." 

Given these criticisms, however, critical theory and the values 
supported by this paradigm form the basis of this proposed 
analytic freunework. 

Contextual considerations . Having explored ideological 
perspectives and social science paradigms, the second layer of 
this evaluation model addresses the broader social, cultural, 
political, and historical context. Decontextualized models, 
simply showing linear relationships between inputs of education 
reform and expected outcomes, have severe limitations. A critical 
reflection demands an awareness of how educational structures, 
contexts, and processes are connected to the social and political 
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forces both inside and outside the school* 

Martin Carnoy and Henry Levin's treatise of the impact of 
the State on educational reform provides an effective means to 
analyze reform in a broader socio-political-economic-cultural 
context. A contextual analysis of education reform is neces- 
sary because educational practices are in reality social practic- 
es and educational reform is social reform. As public institu- 
tions, whose purpose is to serve a varied constituency with 
differentiated expectations, any analysis disregarding this 
larger, broader context would be amiss as Martin Carnoy and Henry 
Levin assert, 

schools in America are largely public institutions. As 
such, they are subject to direct political pressures 
that are conditioned by the overall conflict between 
capital and labor, by the changing structure of the 
labor market, and by various social movements seeking 
greater equality. 

Schooling, afterall, does not merely exist to serve the values of 
education, but also serves particular social interests and the 
institutionalized self-interests of particular groups. Failure 
of educational reforms then, is largely due to a lack of v.nder- 
standing regarding the nature of educational change and its 
relationships with the socio-economic environment or avoiding 
specification of "external" factors that lead to conflict over 
reform priorities. A contextual analysis, thus, helps account for 
the seeming contradictions in the various education reports 
coming out of the 80 's with some calling for reform efforts aimed 
at efficiency, some for excellence and some for equity. 

Broad contextualized conceptualizations of education refom 
are criticized, however, from the perspective that they consist 
of macro analyses of education and schooling. Macro approaches 
often regard human beings as little more than products of social- 
ization with human creativity seemingly ignored and human freedom 
non-existent. According to Eric Bredo and Walter Feinberg, "macro 
approaches fail to grasp the reality of life in schools and do 
not help us understand what makes teachers and pupils 'tick'."^'' 
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Components of the model * while impact is generally the end 
process of a reform effort^ any framework analyzing impact also 
needs to be concerned with the steps and processes leading up to 
impact^ as those will definitely influence the impact of the 
reform effort. Thus^ a model must include an analysis comparing 
the objectives of the reform to the processes by which the reform 
is conceived^ formulated^ and implemented. Richard Sack proposes 
such a two-dimensional conceptualization of reform which includes 
questions regarding both the objectives of the reform and pro- 
cesses of the reform. A third category regarding the outcomes 
of the reform needs to also become a part of such a model. An 
adaptation of Frank Fischer's model for doing critical research 
is included in this framework. 

fl) Objectives of the reform . In looking at objectives (or 
planned outcomes) of reform^ Sack discerns between two major 
types: (1) those which are conceived and defined in terms specif- 
ic to the educational system itself and (2) those which are con- 
ceived and defined in terms external to the educational sys- 
tem. Examples of objectives falling within the first category 
could comprise pedagogical reforms intended to improve teaching 
and learning practices or reforms intended to increase efficiency 
measures within schools. Reforms of an external nature comprise 
examples such as the development of a better articulation between 
the education system and the employment structure of the economy 
or the democratization of the school system to eliminate class, 
race or gender inequalities. 

A logical point of departure regarding objectives is to 
analyze them against the broader context within which the refoirm 
is situated. The scope of this analysis is dependent on whether 
the reform effort is federal or state legislation, local initia- 
tives, or prograuiunatic initiatives (such as calls for reform from 
the National Council of Teachers of Mathematics). This broad- 
based analysis is important because having a more in-depth 
understanding of the reform (i.e. the purpose of the reform, why 
it came about, the need for the reform, its intended outcomes. 



etc.) has the potential to open up the realm of possibilities and 
alternative strategies that a district or school can pursue in 
actually meeting compliance or in implementing said reform. 

Asking basic questions about the objectives of school reform 
is critical, also, because the objectives of school reform are 
often denied by discrete programs or are kept purposely cimbigu- 
ous. In the example of the current push for excellence, an 
assumption is made by policymakers that if the various program 
objectives are realized, educational excellence naturally fol- 
lows. At the same time, "there is no definition of excellence, no 
concept of what excellent schools, as institutions should look 
like."^^ Therefore, the task of evaluating those outcomes be- 
comes a matter of evaluating criteria which are more prone to 
measuring, but what is measurable often has little relation to a 
program's original objectives." 

In considering the objectives of the reform within the 
broader context, an even more fundamental question needs to be 
asked: "what is the perceived purpose of schooling and how does 
this particular reform fit within that purpose?" This is an 
important consideration because differing perceptions of school- 
ing purposes (i.e. socialization vs. sorting functions vs. 
empowerment) will impact on the adoption and acceptance of 
reforms depending on the congruence between the reform and the 
felt need^^. 

(2) Processes of the reform . The second category which forms 
the basis of this framework focuses on the processes by which the 
reform is conceived, formulated and implemented in determining 
the impact of the reform. This is a key element because the 
strategies used regarding these processes are central to the 
outcomes of the reform effort. Processes include four categories: 
(1) level and nature of participation, (2) flow of information 
both upward and downward, (3) decision-making processes, and (4) 
resource allocation processes. 

(a) Level and nature of participation . In analyzing the 
impact of the reform using participation measures, it is critical 
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to look at the level and nature of participation during both the 
conception and implementation of the reform^ A critical part of 
this component is also looking at who participates in the pro- 
cess. The concern has typically been for teacher involvement in 
initiating reform efforts^ but it is equally crucial for princi- 
pals and other educational leaders within the district to be 
visibly involved as well.^^ In addition to the traditional peo- 
ple in the educational setting^ more collaboration is needed^ 
both within the education establishment and beyond to include 
parents^ legislators^ governors and the community as a whole 
Regarding curricular reform issues^ Shirley Grundy suggests that 
students^ also^ should be a vital part of that process; not only 
on the receiving end but involved in the critical decision-making 
operation as well." A critical perspective would look at who 
was involved in the process and ask why particular groups were 
either involved or omitted from the process. 

In any reform effort^ it is crucial to involve teachers in 
all phases of the reform^ from planning through implementation. 
According to the Rand Study, teacher involvement enhanced local 
commitment and motivation as well as built capacity to use an 
innovation. The literature stresses that local participation 
in project planning increases the possibility that changes will 
be appropriate for the particular setting. 

The nature of participation seeks answers to how people feel 
about becoming involved in the reform effort; did they partici- 
pate willingly or was participation mandated? The nature of 
participation can have profound effects on the eventual outcomes 
of the innovation.^^ In an evaluation of federally funded change 
agent prograuns^ Paul Herman and Milbrey McLaughlin categorized 
reason for involvement as either opportunism (becoming involved 
due to available funds) or problem-solving (becoming involved dira 
to a felt need). They found lack of interest and commitment to 
projects generated essentially by opportunism as compared to 
strong commitment for problem-solving projects.^^ 

Based on his research of a school district piloting PPBS 
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(Planning Programming Budgeting System), Harry Wolcott concludes 
that teacher's perceptions of the usefulness of the innovation 
will impact on the nature of their involvement,^^ He concludes 
that compliance doesn't necessarily ensure enthusiastic support 
nor does resistance imply total lack of involvement, but rather, 
there are degrees of participation as well as degrees of resis- 
tance • From a critical perspective, the issue of rejjistance could 
be attributable to feelings of alienation or deskilling which 
sometj nes occur in prepackaged reform efforts."*^ Thus, the 
source and reasons for resistance need to be ferreted out very 
carefully* 

(b) Flow of information both upward and downward > This cate- 
gory refers to the communication process both to and from the 
school regarding the reform effort. In the case of planning for 
reform, Richard Sack says, 

the flow of information is particularly important since 
reforms exist in relationship to some perceived need 
for change whose nature, existence, and intensity are 
conveyed to planners by the quality and availability of 
the information they receive 

In looking at the flow of infomation consideration needs to be 
given to the quality of the communication process between the 
organization (government, private, research, etc,) mandating the 
reform and the school involved in implementation. It must also 
consider the best method to provide needed and relevant informa- 
tion regarding the reform and of what that should consist. 

In the Rand Study, Paul Berman and Milbrey McLaughlin found 
that top-down directives, even when issued with the best inten- 
tions, generally met with indifference or resistance at the 
school level.*" David Marsh and Gregory Bowman, on the other 
hand, argue that top-down content-oriented strategies are effec- 
tive for implementing comprehensive reformis, but rc4uire bottom- 
up participation to be effective.** Bottom-up process-oriented 
reform strategies were found to be more effective for implement- 
ing unique programs focusing on specific student populations.*^ 

Staff development needs b^opme a consideration within the 
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information flow component. In studying the impact of staff 

development on education reform, Nancy Rennau Tomposky says, 

it appears that teachers' dispositions to deal favor- 
ably or not with proposed curriculum changes are (sic) 
related to how they construe their roles in the educa- 
tional process. When this perception is at odds with 
the teacher role envisioned by curriculum designers, 
the role mismatch can be the source of resistance to 
implementation . *® 

Consequently, policy-makers see staff development primarily as a 
technical service rather than as a learning experience for all 
the participants. Those providing staff development typically 
focus their efforts on skills training through demonstration and 
practice, rather than giving adequate attention to the conceptual 
underpinnings of a given innovation. What is needed is staff 
development which is : 

1. ongoing and interactive wherein participants can discuss their 
perceptions of the change process during its different phases 
of implementation; 

2. provided by teachers themselves; 

3. designed by the teachers, themselves where they can select, 
define, and solve problems as they arise rather than being 
organized around predetermined topics generated by profes- 
sional reformers or academicians; and 

4. formative in nature, i.e. an ongoing assessment of curriculum 
as it is being implemented.*^ 

In studying science education reforms in New York, E. Wayne 
Ross, Nathan Aubrey, Christine Berte, and June Cohen examine the 
contradictions which exist within a system stressing measurement- 
driven curriculum and teacher professionalism at the same 
time.^° While the issue of teacher professionalism advocated 
teacher dialogue with policy-makers on the reform, teachers, for 
the most part, felt they didn't have a voice in the reform 
itself. Instead they were primarily perceived as passive recipi- 
ents of a reform effort and their role was reduced to that of 
technicians .^^ 

A critical reading of this issue may involve asking ques- 
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tions about the origin of the reform and the role of teachers in 
the process; are they merely passive recipients of information 
and considered as technicians in carrying out the reform or are 
they empowered to make critical decisions regarding the implemen- 
tation of the reform? 

(c) Decision-making processes . In examining this process^ 
two modes of decision-making need to be considered: (1) central- 
ization vs. decentralization and (2) democratic vs. technocratic 
decision-making. The role of decision-making on the impact of 
reform is illustrated in three case studies of state-initiated 
reform in Texas ^ California and South Carolina." Texas used a 
highly centralized and technocratic^ rational planning approach 
to implementing state reform efforts^ California a laissez faire 
approach and South Carolina an extremely democratic approach 
allowing for input from nearly 40^000 persons state-wide. Thomas 
Timar and David Kirp assert that "the distinguishing attributes 
of (South Carolina's) strategy are consensus on policy goals, 
discretionary authority in implementation, and a strong commit- 
ment to state goals/* which ensured a much higher degree of 
success than the other two states 

Regardless from whichever level the reform is emanating 
(i.e. national, state or local) the site of decision-making can 
ultimately effect impact of the reform. "The tension between 
increased state regulation and the need to maintain local flexi- 
bility poses a fundamental dilemma in the effort to implement 
excellence" or any reform effort.^^ 

Questions regarding decision-making processes from a criti- 
cal perspective include: (1) how do differences in management 
processes effect policy outcomes and what implications do they 
have on issues of control, (2) who needs to be involved in making 
decisions and how does this involvement contribute to ownership, 
or how does lack of involvement lead to alienation, and (3) how 
can democratic processes of decision-making pose less of a threat 
to those in le^idership roles whose current positions are tied to 
the existing power structure within a school? 
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(d) Resource allocation processes . Resources includes both 
material and human resources. More than any other local condition 
and its impact on how reform strategies are enacted in a school 
is the concern regarding availability of school leHources. "If 
staff time and the money to purchase staff time and materials are 
scarce, it is unlikely that change activities will make much, if 
any, headway."" Source of resources also is a major consider- 
ation; whether federal money or other sources of money are 
available and consequent continuation of the project when funds 
run out. 

Questions bringing a critical perspective to the resource 
category include: (1) what is the basis of resource allocation, 
(2) what kinds of reforms typically get funding, (3) which 
interest groups in society are getting funding for their reform 
proposals and which are not, and (4) what or who is determining 
prioritization for funding. 

(3^ Outcomes of the reform ^ The final category of key ques- 
tions focuses on the outcomes of the reform effort. The rational 
systems approach uses a linear model in implying that improve- 
ments in educational outcomes (the immediate effects of school- 
ing) should lead to improved social outcomes (changes in the 
society). A reading of the literature, however, points to the 
fact that this relationship is not necessarily linear; an im- 
provement in one doesn't necessarily lead to an improvement in 
the other. Therefore, other models, such as Frank Fischer's 
critical theory model warrant consideration to determine the 
outcome of a particular reform effort.^® 

A model based on critical theory supports the necessity to 
use standards of societal concerns in assessing to what degree 
educational outcomes have been achieved. As mentioned earlier, 
the values of social justice and equity become the standard 
through which impact of education reform is evaluated. Assessing 
the degree of achievement of educational outcomes is accomplished 
through two levels of analysis. The first level of questioning 
connects specific policy goals and outcomes to the larger society 

^ 1 Pi 
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(more of a functionalist approach to doing evaluation) Ques- 
tions at this level include: (1) do program outcomes contribute 
to the efficiency of dominant social practices and institutions 
and (2) are program outcomes politically acceptable*^^ Cleo 
Cherryholmes says that "analysis at this level remains fundamen- 
tally instrumental, and dominant social values are accepted as 
valid." The second level of analysis questions the criteria 
and standards that previously had been accepted as givens* 
Consequently, questions could include: (1) are the outcomes 
equitable and just; (2) do they promote the advantage of all? 
and, if not, (3) is it possible to alter social practices and 
institutions to further the advantage of all. Other questions 
could be: (1) whose interests are served or who benefits from the 
reform effort, (2) does the reform reinforce the status quo or 
assist in challenging the status quo, and (3) does the reform 
increase the quality of life of persons who are involved in the 
education process and lead to transformation of individuals or 
society* Although the questions will vary, the values serving as 
a filter or criteria for assessing the reform emanate from a 
critical perspective. Michael Apple and Landon Beyer believe that 
a model for doing program evaluation must include a process of 
reflection about the interests, values, and ideologies in the 
program and not merely become a comparison between the stated 
goals of the program and how closely those goals have been met, 
as is often the case with technically sophisticated models ♦^^ 

Summary 

The purpose of this piece has been to formulate an analytic 
model to evaluate the impact of education reform. The impetus for 
this piece has emanated from five years of experience with 
schools as an evaluation consultant assisting educators to manage 
and cope with educational changes and reforms initiated or 
mandated either locally, by the state, or via federal legisla- 
tion. In assisting educators to look beyond the mere implementa- 
tion of reform goals and to situate the reform within broader 
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contexts, this model has enabled me to assist them in deconstruc- 
ting the refom and implementation processes and has made possi- 
ble their understanding regarding the acceptance or rejection of 
reform efforts* Achieving this understanding has proven to be 
empowering to teachers. However, much work still needs to be done 
to enable teachers to reflect on how their understanding of and 
involvement in school reform enables children to become transfor- 
mative intellectuals and society to become more just and equita- 
ble. 
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